Skip to main content

Envoy

What is Envoy?

Envoy is an open source edge proxy created at Lyft, designed to be used with cloud native applications. Envoy adds resilience and observability to your services, and it does so in a way that’s transparent to your service implementation. It's one of the top-rated open-source products targeting issues that a lot of developers at companies like Microsoft, Netflix, SAP, VMware, and Uber are struggling to solve.

download

Envoy not only solves issues, but it also offers some pretty handy features to ease debugging as well as development.

The project was born out of the belief that:

The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem.

In practice, achieving the previously stated goal is incredibly difficult. Envoy attempts to do so by providing the following high level features:

Out of process architecture: Envoy is a self contained process that is designed to run alongside every application server. All of the Envoys form a transparent communication mesh in which each application sends and receives messages to and from localhost and is unaware of the network topology. The out of process architecture has two substantial benefits over the traditional library approach to service to service communication:

  • Envoy works with any application language. A single Envoy deployment can form a mesh between Java, C++, Go, PHP, Python, etc. It is becoming increasingly common for service oriented architectures to use multiple application frameworks and languages. Envoy transparently bridges the gap.
  • As anyone that has worked with a large service oriented architecture knows, deploying library upgrades can be incredibly painful. Envoy can be deployed and upgraded quickly across an entire infrastructure transparently.

As the world is moving towards a distributed microservice architecture, practitioners are observing difficulties in running operations in two main areas:

  • Networking
  • Observability

Envoy is most comparable to software load balancers such as NGINX and HAProxy, but it has many advantages than typical proxies. It allows SSL by default, it is really flexible around discovery and load balancing the workload. Developed at Lyft, Envoy now has a vibrant OS contributors community and is an official CNCF graduated project, along with Kubernetes and Prometheus. Envoy provides more visibility into the system. It's deployed as a sidecar along with service, there is no need to change application logic. Envoy allows Microservices to talk to each other in a transparent, elegant and resilient way and adds observability which is need of the hour for modern Cloud-native web applications. Envoy supported by Google, IBM, and a bunch of other big players.

Why Envoy Proxy Matters?

When there are many traditional proxies then why need a modern Edge proxy such as envoy? NGINX, HAProxy, and Envoy are all battle-tested L4 and L7 proxies, but Envoy has the following additional benefits:

  • It’s developed by keeping modern Microservices in mind.
  • It translates between HTTP-2 and HTTP-1.1.
  • It proxies any TCP protocol.
  • It proxies any raw data, web sockets, databases, etc.
  • It enables SSL by default.
  • It has in-built Service Discovery as well as Load Balancing.
  • It does all the configuration dynamically, all the hosts added dynamically, not like writing the list to a static config file & re-reading it.
  • Envoy stores the mapping of requests from clients (i.e., URLs) to services and the in-built load balancer reads dynamically.

Installing Envoy

The Envoy project provides a number of pre-built Docker images for both amd64 and arm64 architectures. Follow this link to install envoy.

If you are installing on Mac OSX, you can install natively with brew.

Once you have installed Envoy, check out the quick start guide for more information on getting your Envoy proxy up and running.

How Does Envoy Work?

  • Network Stack:- Envoy works at the TCP level: Layer 3/4 Network/Transport proxy. It just read/write bits, and uses IP addresses and port numbers to make a decision about where to route the request. Working at the TCP level is drastically fast and simple. Envoy also works at L7 as well simultaneously when it has to proxy different URLs to different backends since it needs application information available only at L7. Working at 3, 4, and 7 layers allow it to cover up all limitations and have good performance.

  • 2 layers of Envoy:- Edge Envoy The standalone instance is a single point of entry to the cluster. All requests from outside first come here & it sends them to internal Envoys. Sidecar Every instance(replica) of an app has a sidecar Envoy, which runs in parallel with the app. These Envoys monitors everything about the application. All these proxies are inside a mesh, which has internal routing with each, side Envoys does health monitoring and let the mesh know whether a service is down or not. All Envoys also gather stats from the application and sends it to a telemetry component (Mixer in Istio). All of the Envoys in Mesh configured differently by modifying Envoy configuration file, according to a particular use case.

Benefits of Envoy Proxy

  • Wickedly fast than old gen proxies.
  • Wickedly scales horizontally which is needed for modern apps.
  • Proxies added/removed dynamically.
  • Allows filtering the request based on many parameters.
  • For Edge Envoys, any number of servers(each of which points to its own array of hosts) and any number of routes added for different proxy URLs, which gives flexibility in Infra management.
  • While for sidecars, Envoy have only one route, and it will proxy to the app running on localhost.
  • Configuring a sidecar proxy is pretty straight-forward, and configuration updated dynamically.
  • Envoy allows DNS, which is easier to remember and any new service instances are added to the DNS dynamically.
  • Envoy manages, observes and works best at L7.
  • Envoy aligns well with Microservices world.
  • Provides features such as resilience, observability, and load balancing.
  • Envoy embraces distributed architectures and used in production at Lyft, Apple, Google.

Best Practises of Envoy

Best Practices while implementing Envoy proxy for Microservices -

  • To take advantage of all features Envoy provides, whole Service Mesh should be set up, including Edge as well Sidecar proxies.
  • Utilize and configure advanced patterns such as Circuit Breaking, Automatic Retries etc.
  • Test deployments using Incremental Blue/Green Deploys Separate deploy from actual production release. At first, deploy new versions side-by-side without taking traffic. Then, try to shift 1% of traffic to the new version and check the metrics in Grafana dashboard. If everything looks good, try increments 10%, 50%, and finally 100 %.
  • Create alerts based on Prometheus or Grafana metrics.
  • Try to create a self-healing infrastructure.
  • Run all Microservices on same port to maintain consistency.
  • Envoy's listener configured to send requests on that port on each Microservices localhost.

Envoy offers critical debugging functionalities for microservice software systems, network transparency, and common best practice communication design patterns.

Using an Envoy proxy makes software systems more transparent, more resilient, easier to develop against, and less complex to debug.

If you feel that Envoy is something you’d like to check out, you can start from the official docs, its GitHub page, or this wonderful YouTube conference video from Matt Klein, an engineer at Lyft, creator and lead maintainer of Envoy.